1. Field of the Invention
The present invention relates to a horizontal line interpolation circuit which functions to increase or decrease the number of horizontal lines through interpolation processing such as an electronic zoom function, by which an image is electronically enlarged, or some standard conversion, and further relates to an image pickup apparatus provided with a horizontal line interpolation circuit which functions to increase or decrease the number of horizontal lines of a picked-up image through interpolation processing.
Recent years' development of the digital signal processing technique has made it easier to increase or the number of horizontal lines of an image through interpolation processing. Accordingly, with the use of this technique, there have been increasing video equipment incorporating the so-called electronic zoom function, by which an image is electronically enlarged, or a standard conversion function by which a conversion is made from the NTSC system to the PAL system some other standard. This is the reason that image-quality enhancement in the electronic zoom function or the standard conversion function is of growing importance.
2. Description of the Prior Art
As an example of the horizontal line interpolation circuit for increasing the number of horizontal lines of an image through an interpolation circuit, the prior art will be described below taking the case of an image pickup apparatus provided with an image enlargement function which is effected by increasing the number of horizontal lines of an image.
Known as a horizontal line interpolation circuit is the television standards converter circuit, a typical example of which is shown in a literature, Journal of the Japanese Television Society, Vol 29., No. 10, pp. 773 to 784, FIG. 3. Shown in FIG. 27 is a typical example of a conventional image pickup apparatus which electronically enlarges an image picked-up by an image sensor with the application of the above-mentioned horizontal line interpolation circuit. In the figure, reference numeral 1 denotes a solid state image sensor; numeral 2 denotes a driving circuit for the solid state image sensor 1; numeral 3 denotes an amplifier for amplifying an output signal from the solid state image sensor 1; numeral 68 denotes a process circuit generating a luminance signal or the like from the output of the amplifier 3; numeral 69 denotes a vertical aperture correction signal generator; and numeral 70 denotes a first adder for obtaining a luminance signal whose vertical sharpness has been corrected by adding a luminance signal from the process circuit 68 and a vertical aperture correction signal from the vertical aperture correction signal generator circuit 69. Numeral 5 denotes an electronic zoom circuit for enlargement-processing an output signal from the first adder 70 by increasing the number of horizontal lines through interpolation processing, the electronic zoom circuit 5 generally comprising: a memory circuit 6; a memory control signal generator 10 for generating control signals for write, read, and addresses of the memory circuit 6; multipliers 7 and 8 for multiplying signals of two lines read from the memory circuit 6 by their respective interpolation weight coefficients w and (1-w); a second adder 9 for adding the output signals of the multipliers 7 and 8 to yield an interpolation output; and an interpolation coefficient generator 11 for generating the interpolation weight coefficients w and (1-w) for signals of two lines in accordance with instructions from a control section 64. The control section 64 generates control signals to the memory control signal generator 10 and the interpolation coefficient generator 11 according to the zoom multiplying factor of the electronic zoom and the screen position to be zoomed (zoom position). Numeral 71 denotes a selector for switching between electronically zoomed output of the electronic zoom circuit 5 and non-zoomed output of the first adder 70 in accordance with instructions from an operation switch 52; the selector 71 feeds a signal to an output terminal 66. Numeral 52 denotes an operation switch for giving instructions for on/off control of the electronic zoom as well as the zoom multiplying factor and screen position to be zoomed (zoom position) for the on-control of the electronic zoom. In addition, for example, the image pickup apparatus which is so arranged as to perform an electronic zoom operation of the same multiplying factor at all times does not require the selector 71 or the operation switch 52; such a conventional apparatus is shown in FIG. 28. As shown in the figure, all but the selector 71 and the operation switch 52 of its construction are the same as in FIG. 27.
In this case, the solid state image sensor 1 is scanned by a normal scan method (a scan method corresponding to an interlace scan), as shown in FIG. 32. More specifically, a signal charge of each pixel of the solid state image sensor 1 is scanned in such a way that for the odd field, as indicated by a solid line in FIG. 32, two adjoining horizontal pixel rows are scanned by one-time horizontal scan, while for the even field, as indicated by a dotted line in FIG. 32, the two pairs of horizontal pixel rows simultaneously read by a one-time horizontal scan are scanned by shifting one row with respect to the odd field in the vertical direction, wherein as to an output signal, the relationship between the scanning lines of the odd field and those of the even field is an interlace relationship, as shown in FIGS. 33 (a)-33(b). In FIG. 33, solid line represents scanning lines of the odd field, and a dotted line represents those of the even field.
Shown in FIG. 29 is a conceptual view of the image enlargement with the above-described image enlarging apparatus. Now assume that the solid state image sensor 1 puts out an image of 240 lines for one field and therefore 480 lines for one frame.
The case is described below where, of these lines, a portion equivalent to 200 lines for one field is enlarged to obtain an image for normal one field (i.e. a signal of 240 lines), as shown in FIG. 29. The multiplying factor is, in this case, 240.div.200=1.2.
To increase the number of scanning lines from 200 to 240, the electronic zoom circuit 5 in FIG. 27 performs an interpolation processing as shown in FIG. 30 (a). In detail, for example, to obtain the (N+1) th line, input lines of the n th and (n+1) th lines are read from the memory circuit 6 and multiplied by interpolation weight coefficients relative to distances (here, 2/12 and 10/12), and then added together. Similarly, other output lines are also obtained from the upper and lower two lines by multiplying interpolation weight coefficients relative to distances and adding together. Here, the vertical frequency response characteristic of the output signal obtained by the interpolation processing is examined. A line interpolated with interpolation weight coefficients of 1/2 and 1/2 results in the perfect average of input two lines, causing the vertical frequency response characteristic to be the lowest, while a line interpolated with interpolation weight coefficients of 1 and 0 allows one line of an input signal to be output as is, causing the frequency response characteristic to be the highest.
In consequence, the vertical frequency response characteristic of each output line can be graphed as shown in FIG. 30 (b). The graph shows that the lines are either high or low in vertical frequency response, causing high and low portions in vertical frequency response to take place on the output screen, as shown in FIG. 31.
Next described is the case where one frame, or a two-field image (a signal of 240+240=480 lines) is obtained by the same image enlargement processing as above. In this case, the electronic zoom circuit 5 in FIG. 27 performs an interpolation processing as shown in FIG. 33 (a). More specifically, odd field output lines are obtained in the odd field from odd field input lines in FIG. 33 (a), while even field output lines are obtained in the even field from even field input lines in the same figure (a). For example, to obtain the (N+1) th line of the odd field, input lines n and n+1 are read from the memory circuit 6, multiplied by interpolation weight coefficients relative to distances (in this case, 2/12 and 10/12), and added together. Similarly, other output lines are also obtained from the upper and lower two lines by multiplying interpolation weight coefficients relative to distances and adding together.
Here, vertical frequency response characteristic of the frame output image signal obtained by the interpolation processing is examined, as with the field image. The result is that the vertical frequency response characteristic of each output line can be graphed as shown in FIG. 33 (b). In the figure, where a solid line represents the characteristic of the odd field and a dotted line represents that of the even field, a pair of adjacent lines in the odd and even fields (e.g. the (N+1) th line of the odd field and the (N+1), th line of the even field) are approximately equal in their interpolation weight coefficients. Thus, the positions of the lines having high vertical frequency responses and those of the lines having low vertical frequency responses coincide with each other between the odd and even fields, as shown in the figure. Accordingly, the positions of lines having high vertical frequency responses and those of the lines having low vertical frequency responses in the output image also coincide with each other between the odd and even fields, as shown in FIGS. 34(a)-34(b).
Next, the vertical noise characteristic of an output signal obtained by the interpolation processing is examined. The vertical noise characteristic of an output signal obtained by the interpolation processing also results in that interpolated line being the perfect average of two input lines, as in the above-described cases, when the lines are interpolated with interpolation weight coefficients of 1/2 and 1/2. Accordingly, the resulting S/N ratio is improved by 3 dB at a maximum, hence a satisfactory S/N ratio. On the other hand, when the lines are interpolated with interpolation weight coefficients of 1 and 0, the input one line is output as is, with the result that the S/N ratio is not improved, hence the worst S/N ratio. In consequence, the noise level characteristic for each output line can be graphed as shown in FIG. 35 (b). This indicates that an electronically zoomed output image has different S/N ratios of lines, causing noise differences between lines to be viewed as disturbing lateral stripes in the output image, as shown in FIG. 36.
As a note to the above description, which has been made on an image pickup apparatus provided with the electronic zoom function (image enlarging function), such a problem that the vertical frequency response and noise characteristics increase depending on vertical positions of the screen is not limited to the above apparatus, but commonly arises in image enlarging apparatus for electronically enlarging images.
However, the conventional horizontal line interpolation circuit as described above has the following problem. As stated before, the vertical frequency response characteristic of an output line varies with interpolation weight coefficients and therefore differs between output lines. As a result, the output screen involves high and low portions in vertical frequency response, causing the vertical frequency response of an image to be diverse to a great extent depending on the vertical positions of the screen. This disadvantageously results in a most undesirable, unfriendly-to-see image.
Also in one frame image made up of two fields, the positions of the lines having high frequency responses and those of the lines having low frequency responses approximately coincide with each other between the odd and even fields, causing high and low portions in vertical frequency response to take place also in the output screen made up of two fields. This results in a most undesirable, unfriendly-to-see image with the vertical frequency response of an image diversified to a great extent depending on the vertical position of an image.
Moreover, the image enlarging apparatus and the image pickup apparatus which perform conventional electronic zoom as described above, as stated above, involve different S/N ratios of lines in an electronically zoomed output image, which appear as disturbing lateral stripes in an output image. Accordingly, in common image enlarging apparatus, especially when input signal level is such low as to require the gain of a gain control amplifier to be increased, the S/N ratio of a signal entered into the electronic zoom circuit is unsatisfactory with disturbing lateral stripes conspicuously present. Further, in the image pickup apparatus, especially when a dark object is picked up which requires the gain of the gain control amplifier to be increased, the S/N ratio of a signal entered into the electronic zoom circuit is unsatisfactory, causing the disturbing lateral stripes to be conspicuously present.